-
Notifications
You must be signed in to change notification settings - Fork 156
x86/fgraph,bpf: Fix ORC stack unwind from kprobe_multi #10846
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: bpf-next_base
Are you sure you want to change the base?
Conversation
|
Upstream branch: 78980b4 |
f416fe6 to
0cc2682
Compare
|
Upstream branch: 95ffdbd |
182c73a to
f25c57c
Compare
0cc2682 to
6797dbd
Compare
|
Upstream branch: 95ffdbd |
f25c57c to
af893bd
Compare
6797dbd to
91d46f6
Compare
|
Upstream branch: 78980b4 |
af893bd to
11cf628
Compare
91d46f6 to
9624cf2
Compare
|
Upstream branch: 8016abd |
11cf628 to
7956bb7
Compare
9624cf2 to
3a73c9c
Compare
|
Upstream branch: 1456ebb |
7956bb7 to
92228fe
Compare
3a73c9c to
aa9aae9
Compare
|
Upstream branch: 35538db |
92228fe to
99e56f8
Compare
aa9aae9 to
f94167c
Compare
|
Upstream branch: 35538db |
99e56f8 to
c6f0506
Compare
f94167c to
cd8cbf1
Compare
|
Upstream branch: 35538db |
c6f0506 to
325b029
Compare
cd8cbf1 to
358bea9
Compare
|
Upstream branch: ae23bc8 |
325b029 to
0f979b5
Compare
358bea9 to
857ca44
Compare
|
Upstream branch: ba22540 |
0f979b5 to
6809a4a
Compare
857ca44 to
32dcdce
Compare
|
Upstream branch: 08a7491 |
6809a4a to
6f50b82
Compare
32dcdce to
ab9b393
Compare
|
Upstream branch: 08a7491 |
6f50b82 to
fd30651
Compare
|
Upstream branch: 08a7491 |
fd30651 to
8983db2
Compare
ab9b393 to
f865ee4
Compare
The previous change (Fixes commit) messed up the rsp register value,
which is wrong because it's already adjusted with FRAME_SIZE, we need
the original rsp value.
This change does not affect fprobe current kernel unwind, the !perf_hw_regs
path perf_callchain_kernel:
if (perf_hw_regs(regs)) {
if (perf_callchain_store(entry, regs->ip))
return;
unwind_start(&state, current, regs, NULL);
} else {
unwind_start(&state, current, NULL, (void *)regs->sp);
}
which uses pt_regs.sp as first_frame boundary (FRAME_SIZE shift makes
no difference, unwind stil stops at the right frame).
This change fixes the other path when we want to unwind directly from
pt_regs sp/fp/ip state, which is coming in following change.
Fixes: 20a0bc1 ("x86/fgraph,bpf: Fix stack ORC unwind from kprobe_multi return probe")
Signed-off-by: Jiri Olsa <[email protected]>
Reviewed-by: Steven Rostedt (Google) <[email protected]>
Mahe reported missing function from stack trace on top of kprobe
multi program. The missing function is the very first one in the
stacktrace, the one that the bpf program is attached to.
# bpftrace -e 'kprobe:__x64_sys_newuname* { print(kstack)}'
Attaching 1 probe...
do_syscall_64+134
entry_SYSCALL_64_after_hwframe+118
('*' is used for kprobe_multi attachment)
The reason is that the previous change (the Fixes commit) fixed
stack unwind for tracepoint, but removed attached function address
from the stack trace on top of kprobe multi programs, which I also
overlooked in the related test (check following patch).
The tracepoint and kprobe_multi have different stack setup, but use
same unwind path. I think it's better to keep the previous change,
which fixed tracepoint unwind and instead change the kprobe multi
unwind as explained below.
The bpf program stack unwind calls perf_callchain_kernel for kernel
portion and it follows two unwind paths based on X86_EFLAGS_FIXED
bit in pt_regs.flags.
When the bit set we unwind from stack represented by pt_regs argument,
otherwise we unwind currently executed stack up to 'first_frame'
boundary.
The 'first_frame' value is taken from regs.rsp value, but ftrace_caller
and ftrace_regs_caller (ftrace trampoline) functions set the regs.rsp
to the previous stack frame, so we skip the attached function entry.
If we switch kprobe_multi unwind to use the X86_EFLAGS_FIXED bit,
we set the start of the unwind to the attached function address.
As another benefit we also cut extra unwind cycles needed to reach
the 'first_frame' boundary.
The speedup can be measured with trigger bench for kprobe_multi
program and stacktrace support.
- trigger bench with stacktrace on current code:
kprobe-multi : 0.810 ± 0.001M/s
kretprobe-multi: 0.808 ± 0.001M/s
- and with the fix:
kprobe-multi : 1.264 ± 0.001M/s
kretprobe-multi: 1.401 ± 0.002M/s
With the fix, the entry probe stacktrace:
# bpftrace -e 'kprobe:__x64_sys_newuname* { print(kstack)}'
Attaching 1 probe...
__x64_sys_newuname+9
do_syscall_64+134
entry_SYSCALL_64_after_hwframe+118
The return probe skips the attached function, because it's no longer
on the stack at the point of the unwind and this way is the same how
standard kretprobe works.
# bpftrace -e 'kretprobe:__x64_sys_newuname* { print(kstack)}'
Attaching 1 probe...
do_syscall_64+134
entry_SYSCALL_64_after_hwframe+118
Fixes: 6d08340 ("Revert "perf/x86: Always store regs->ip in perf_callchain_kernel()"")
Reported-by: Mahe Tardy <[email protected]>
Signed-off-by: Jiri Olsa <[email protected]>
Acked-by: Steven Rostedt (Google) <[email protected]>
We now include the attached function in the stack trace, fixing the test accordingly. Fixes: c9e208f ("selftests/bpf: Add stacktrace ips test for kprobe_multi/kretprobe_multi") Signed-off-by: Jiri Olsa <[email protected]>
Adding test that attaches kprobe/kretprobe and verifies the ORC stacktrace matches expected functions. The test is only for ORC unwinder to keep it simple. Signed-off-by: Jiri Olsa <[email protected]>
Adding test that attaches fentry/fexitand verifies the ORC stacktrace matches expected functions. The test is only for ORC unwinder to keep it simple. Signed-off-by: Jiri Olsa <[email protected]>
|
Upstream branch: 95dbe21 |
Adding support to call bpf_get_stackid helper from trigger programs, so far added for kprobe multi. Adding the --stacktrace/-g option to enable it. Signed-off-by: Jiri Olsa <[email protected]>
8983db2 to
bf46b41
Compare
Pull request for series with
subject: x86/fgraph,bpf: Fix ORC stack unwind from kprobe_multi
version: 1
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=1047287